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The purpose of this study was to examine teachers’ implementation of a specific mathematics 
curriculum and how their conceptions influenced this implementation. An innovative, powerful 
tool, known as “teacher logs,” was used to examine teachers’ implementations of the 
curriculum, a survey was used to study teachers’ conceptions related to their implementations. 
Results suggest that teachers tend to implement the materials closely to the written text, but there 
was variety shown both within and among teachers. A relationship was shown between teachers’ 
implementation levels and their beliefs about mathematics and their beliefs about students. The 
results of this study reveal that teachers may or may not stray from the authors’ intended 
curriculum, and whether or not they stray often relates to their conceptions. 
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When new curricula are developed, researchers, policy makers, and teachers often ask, “Do 
these materials work? Are these materials effective?” As a result, a committee was charged by 
the National Research Council (NRC) to determine the quality of curriculum evaluations. In their 
report, On Evaluating Curricular Effectiveness (NRC, 2004), the committee defined curricular 
effectiveness as, “how effective a particular curriculum is, and for whom and under what 
conditions it is effective” (p. 1). When determining curricular effectiveness, one important part to 
consider is teachers’ faithfulness to the curriculum authors’ intentions because if teachers do not 
implement the materials as the authors intended, they may undermine the effectiveness of the 
new curriculum. Specifically, implementation fidelity needs to be considered. Holstein, Dietz, 
and Keene (2010) defined implementation fidelity as “the extent to which a teacher implements a 
curriculum as the author intended, allowing for adaptation, supplementation, and improvisation 
by the teacher” (p. 2). The concept of implementation fidelity is not equivalent to teacher quality. 
That is, low implementation fidelity does not suggest poor teaching. Rather, researchers study 
implementation fidelity to determine the extent to which the teacher implemented the materials. 
This information can then be used, in part, to determine curricular effectiveness. 

Recent research on implementation fidelity has emerged partly because teachers often 
transform the written curriculum into a form completely different from the authors’ original 
intentions. One may ask, “Why does this transformation occur?” One explanation is teachers’ 
conceptions (e.g., about math, teaching, students, or curricula) influence their instructional 
decisions. Therefore, when examining implementation fidelity, it is important to consider 
teachers’ conceptions. We use the word “conception” in the way Lloyd and Wilson (1998) used 
it: “to refer to a person’s general mental structures that encompass knowledge, beliefs, 
understandings, preferences, and views” (p. 249) 


Context and Purpose of the Study 
The context for this study is the NSF-funded Mathematics for Decision-Making (MDM) 
curriculum (Project DRL-0733137). The emphasis of this curriculum is on decision-making 
using mathematical models based on engineering concepts. The text contains large real-world 
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problem contexts that require multiple steps to solve and interpretation of the solution. The 
curriculum is intended for high school seniors who have completed two years of algebra and one 
year of geometry (or the equivalent). 

The purpose of this study was to examine teachers’ implementations of the MDM curriculum 
and how their conceptions influence their implementations. In particular, we studied how closely 
teachers’ enacted curriculum aligned with the authors’ intended curriculum. We also investigated 
teachers’ conceptions that could bring their enacted curriculum closer or farther from the 
authors’ intended curriculum. The particular conceptions we considered were teachers’ (a) 
subject matter knowledge, (b) beliefs about mathematics, (c) beliefs about students, (d) beliefs 
about teaching, and (e) beliefs about curricular materials. 

The research questions were as follows: 

1. How do teachers’ implementations of a mathematical decision-making curriculum align 

with the authors’ intentions? 

2. How do teachers’ conceptions influence their implementations of a mathematical 

decision-making curriculum? 


Research Background 

The NRC (2004) developed a framework to define “what is meant by a scientifically valid 
evaluation study for reviewing mathematics curriculum effectiveness” (p. 36). It included three 
major components for curriculum evaluations: “(1) the program materials and design principles; 
(2) the quality, extent, and means of curricular implementation; and (3) the quality, breadth, type, 
and distribution of outcomes of student learning over time” (p. 4). Scientifically valid evaluation 
studies on curricular effectiveness should include all the components of this framework. In the 
present study, we examined the “Implementation Components” (the second major component) 
piece of the framework, with the understanding that future studies should include the other 
components of the framework to determine the overall effectiveness of the MDM curriculum. 

One way to study the implementation components of a curriculum is to examine how the 
curriculum transforms from the written materials to what students actually experience in the 
classroom. Stein, Remillard, and Smith (2007) provided a framework for studying the transitions 
of the curriculum from its written form to its enacted form, as shown in Figure 1. 
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Figure 1: Temporal Phases of Curriculum Use (Stein et al., 2007, p. 322) 
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The curriculum begins as a written curriculum, which refers to the authors’ original printed 
materials. Then, the curriculum changes into what the teacher intends to teach. This initial 
transformation occurs due to teachers’ conceptions, goals, and contexts, as shown in the oval in 
Figure 1. Next, the curriculum transforms again into the enacted curriculum, which is the 
classroom implementation of curriculum materials. The enacted curriculum is also influenced by 
outside factors. Finally, student learning occurs. Therefore, it is unreasonable to measure the 
connection between the written curriculum and student achievement without considering the 
transformations of the curriculum. In other words, to determine whether a curriculum is 
effective, one must study whether the teacher is using the curriculum as intended by the 
curriculum authors. 

To capture the implementation and to connect it to the authors’ intentions, Heck, Chval, 
Weiss, and Ziebarth (2012) created two constructs: (a) the mathematical storyline, which refers 
to “which topics are addressed, at what depth different topics are addressed, and/or what learning 
goals and student expectations are pursued” (p. 69) and (b) the pedagogical storyline, which 
includes “which activities are conducted and how they are conducted in terms of openness or 
guidance, how students are grouped for instructional activities, and whether the patterns of 
interaction that are designed are actually pursued” (p. 70). These two constructs guided the 
methodology of this study (described below). 

Moreover, one reason the transformation from written curriculum to enacted curriculum 
occurs is teachers’ conceptions. Many conceptions influence teachers’ instructional decisions 
and thus their implementation fidelity. Five conceptions which consistently emerged in the 
literature as influencing teachers’ instructional decisions were examined in this study: (a) subject 
matter knowledge (e.g., Manouchehri & Goodman, 1998), (b) beliefs about math (e.g., Collopy, 
2003), (c) beliefs about teaching (e.g., Stipek, Givvin, Salmon, & MacGyvers, 2001), (d) beliefs 
about students (e.g., Arbaugh, Lannin, Jones, & Park-Rogers, 2006), and (e) beliefs about 
curricular materials (e.g., Frykholm, 2004). 


Methodology 

In this paper, we present the quantitative portion of a mixed-methods study (see Holstein, 
2012). To answer the first research question, an innovative tool called “teacher logs” was used. 
To answer the second research question, a survey was used, where the survey results were 
compared to the teacher log results. The participants came from the population of high school 
mathematics teachers who attended an MDM workshop (N = 232). Thirteen teachers completed 
at least one teacher log; 12 teachers completed the survey and at least one teacher log. 
Teacher Logs 

Huntley (2009) stated, “If we truly want to understand the relationships among textbooks, 
teaching, and student learning, it is critically important to develop curriculum-sensitive measures 
of implementation” (p. 357). In this study, we used teacher logs (Heck et al., 2012) as our 
“curriculum-sensitive measures of implementation.” Teacher logs were developed by the first 
author of this paper for five chapters in the MDM curriculum (one teacher log per problem 
context and approximately three problem contexts per chapter, for a total of 16 logs). Each log 
contained a list of items for teachers to choose from, where each item illustrated an instructional 
choice. Every effort was made to include all possible implementation choices. Teachers checked 
off items that they did during their implementation of the curriculum, where each item (a) 
supported the authors’ intentions, (b) went against the authors’ intentions, or (c) was neutral. For 
example, the MDM curriculum authors included several in-text questions for students to answer 
themselves, either individually, in groups, or as a class. Therefore, the following item went 
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against the MDM authors’ intentions: “I led a class discussion that covered the information out 
of in-text questions #1-2 (but did not explicitly ask these questions).” On the other hand, the 
following item supported the authors’ intentions: “Students read and answered in-text questions 
#1-2 out loud or individually.” 

Teachers received four scores for each log: (a) a mathematical adherence score, (b) a 
mathematical focus score, (c) a pedagogical adherence score, and (d) a pedagogical focus score. 
The two mathematical scores were based on items that either supported or went against the 
curriculum authors’ mathematical storyline; the two pedagogical scores were based on items that 
either supported or went against the curriculum authors’ pedagogical storyline. 

The mathematical and pedagogical adherence scores refer to how the teacher supported the 
authors’ intentions. These scores were calculated using the following ratio, which yielded a score 
between 0 and 1: 

Number of fideliny elements enacted 
Total number of tidelity elements 
A high mathematical adherence score means that the teacher addressed most of the content of the 
curriculum, while a low mathematical adherence score means that the teacher addressed little of 
the content of the curriculum. A high pedagogical adherence score means that the teacher 
addressed most of the instructional strategies of the curriculum, while a low pedagogical 
adherence score means that the teacher addressed little of the instructional strategies. 

The mathematical and pedagogical focus scores took into account the fidelity elements as 
well as the items that went against the authors’ intentions. These scores were calculated using the 
following ratio, which yielded a score between 0 and 1: 

7 Number of fideliry elements enacted _ Number ot other elements enacted 

Total number of tidelity elements Total number of other elements 
< 

A high mathematical focus score means that the teacher addressed more of the content of the 
MDM curriculum than other content, while a low mathematical focus score means that the 
teacher addressed less of the content of the MDM curriculum than other content. A high 
pedagogical focus score means that the teacher addressed more of the instructional strategies of 
the MDM curriculum than other instructional strategies, while a low pedagogical focus score 
means that the teacher addressed less of the instructional strategies of the MDM curriculum than 
other instructional strategies. Notice that the adherence score is one piece of the focus score. 

Each teacher’s mathematical adherence, mathematical focus, pedagogical adherence, and 
pedagogical focus scores were averaged. In the end, each teacher had four scores, regardless of 
how many logs they completed. These scores informed us about teachers’ levels of 
implementation fidelity. 

Survey 

The purpose of the survey was to gather data on teachers’ conceptions. The survey items 
were adapted from previously validated surveys, where each item measured one of the five 
conceptions included in this study: (a) beliefs about curriculum (items adapted from Spielman & 
Lloyd, 2004), (b) beliefs about math (items adapted from Fennema & Sherman, 1976; Perry, 
Howard, & Tracey, 1999; Suinn, 1988), (c) beliefs about teaching (items adapted from Perry et 
al., 1999; Spielman & Lloyd, 2004), (d) beliefs about students (items adapted from Fennema & 
Sherman, 1976; Suinn, 1988), and (e) subject matter knowledge (items heavily adapted from 
McDiarmid & Wilson, 1991). Note that the “beliefs about curriculum” survey items asked 
teachers about their beliefs about general curricula and were not specific to the MDM 
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curriculum; the “beliefs about math” survey items portrayed mathematics on a traditional to 
nontraditional continuum (Raymond, 1997). All items were on a five-point Likert scale, with the 
exception of the subject matter knowledge items, which were multiple choice. 

Teachers received five scores on the survey (one for each conception). Then, a Spearman’s 
Rank Order correlation was conducted to determine any correlations between the five survey 
scores and the four teacher log scores. 

Results 

In this section, we present the results for the two research questions. First, the teacher log 
results show teachers’ implementation fidelity levels during their implementations of the MDM 
curriculum. Second, the correlations between the teacher log results and the survey results are 
offered to illustrate teachers’ implementation fidelity levels related to their conceptions. 
Implementation Fidelity 

Averages for the four teacher log scores were calculated for each of the 13 teachers, 
regardless of the number of logs completed. Table 1 shows the overall results from these logs. In 
this table, the average score and standard deviation are given for each teacher for each of the four 
scores. This table also gives the following overall data: (a) the average of the teacher averages, 
found by averaging the average scores for the 13 teachers; (b) the standard deviation of the 
teacher averages, found by calculating the standard deviation of the average scores for the 13 
teachers; (c) the minimum teacher average score; and (d) the maximum teacher average score. 


Table 1: Teacher Log Results 


TeAChAe Number of Math Math Pedagogical Pedagogical 
Noniher Completed | Adherence Focus Adherence Focus 
Logs Score SD Score SD Score SD Score SD 
T1 12 0.5822 | (.1529) | 0.7179 | (.1238) | 0.5451 | (.2428) | 0.6867 | (.1640) 
T2 fi 0.6638 | (.1933) | 0.8149 | (.0999) | 0.7794 | (.1969) | 0.8897 | (.0985) 
T3 4 0.7572 | (.1478) | 0.7328 | (.0898) | 0.8333 | (.2357) | 0.8479 | (.0851) 
T4 1 0.9375 | (--) | 0.9688 | (--) | 0.8333 | (--)_ | 0.9167 | (--) 
T5 4 0.9480 | (.0392) | 0.8832 | (.1078) | 0.8294 | (.2243) | 0.8678 | (.0884) 
T6 1 0.8889 | (--) | 0.9444 | (--) 1 (--) 1 (--) 
T7 8 0.7284 | (.1466) | 0.7749 | (.1398) | 0.6217 | (.2033) | 0.6949 | (.1829) 
T8 7 0.7215 | (.1609) | 0.7307 | (.1434) | 0.6643 | (.3473) | 0.7388 | (.2407) 
T9 2 0.5729 | (.3388) | 0.7388 | (.2368) | 0.5 | (.2357) | 0.6688 | (.0913) 
T10 1 0.9444 | (--) | 0.7103 | (--) | 0.8889} (--) | 0.9444 | (-) 
Tl1 4 0.8547 | (.1219) | 0.8023 | (.0916) | 0.9069 | (.1114) | 0.8991 | (.0837) 
T12 2 0.9444 | (.0786) | 0.7961 | (.1116) 1 (0) 1 (0) 
T13 1 0.9444 | (--) | 0.6627| (--) | 0.5556] (--) | 0.7465 | (-) 
eNotes OLE Sarier 0.8068 0.7906 0.7660 0.8386 
Averages 
iad vacesen naa 0.1427 0.0925 0.1714 0.1182 
Averages 
Minimum 0.5729 0.6627 0.5 0.6688 
Maximum 0.9480 0.9688 1 1 


To help make sense of these data, consider two teachers in Table 1| (recall that all scores were 
between 0 and 1). First, T1 adhered to only about half of the mathematical and pedagogical 
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intentions (mathematical adherence score of 0.5822 and pedagogical adherence score of 0.5451), 
but this teacher tended to use the MDM content and instructional strategies more than other 
content or instructional strategies (mathematical focus score of 0.7179 and pedagogical focus 
score of 0.6867). Second, T3 had relatively high scores, but the pedagogical scores (0.8333 and 
0.8479) were higher than the mathematical scores (0.7572 and 0.7328). Thus, this teacher tended 
to be slightly less faithful mathematically than pedagogically. 

When looking at the overall averages, the pedagogical adherence had the lowest average 
(0.766); this means that teachers did not necessarily use the instructional strategies intended by 
the authors and that they only used approximately three quarters of the instructional strategies 
presented on the teacher logs. However, the pedagogical focus score had the highest average 
(0.8386); thus, teachers used other instructional strategies less than they used the strategies 
intended by the MDM curriculum authors. The individual teachers’ standard deviation values 
show variety within teachers; the overall minimum, maximum, and standard deviation values 
show that there was variety among teachers. 

Conceptions Related to Teachers’ Implementation Fidelity 

Each teacher received a score for each construct based on the five conceptions (beliefs about 
curriculum, beliefs about math, beliefs about teaching, beliefs about students, and subject matter 
knowledge). From these data, a Spearman’s Rank Order correlation was conducted to examine 
correlations between the teachers’ implementation fidelity scores on the teacher logs and their 
conception scores on the survey. Table 2 shows the results from this analysis, including the 
correlation coefficients and the p-values (in parentheses). 


Table 2: Correlations between Implementation Fidelity and Conceptions 


Beliefs about | Beliefs about | Beliefs about | Beliefs about Subject 
: ‘ Matter 
Curriculum Math Teaching Students 
Knowledge 
Mathematical en 
Adherenee 194 (.545) | .629** (.028) | .072 (.824) .127 (.694) .051 (.875) 
aes nae .111 (.730) .004 (.991) -.046 (.888) .077 (.811) -.462 (.130) 
Pedagogical j - ; 
Aahiceonee 301 (.342) .327 (.299) .231 (.471) | .659** (.020) | -.268 (.399) 
eas 26 (.167) | .357(.255) | .054(.866) | .536* (.072) | -.317(.315) 


ty <1. ** p< 05. 


The test revealed that there was a statistically significant correlation (p < .1) between 
mathematical adherence and beliefs about math, between pedagogical adherence and beliefs 
about students, and between pedagogical focus and beliefs about students. All other relationships 
were not significant. 

The results for each conception are discussed here. First, there were no significant 
correlations between teachers’ beliefs about general curricula and their teacher log scores. 
Second, teachers’ beliefs about math were significantly positively correlated only with their 
mathematical adherence scores. That is, teachers with more nontraditional views of math tended 
to adhere to the authors’ mathematical storyline. Third, teachers’ beliefs about students were 
significantly positively correlated with their pedagogical adherence and focus scores. That is, 
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teachers with positive views of students and their abilities tended to enact most of the 
instructional strategies of the MDM curriculum (high pedagogical adherence score) and to enact 
more of the instructional strategies of the MDM curriculum than other strategies (high 
pedagogical focus score). Fourth, there were no significant correlations between teachers’ beliefs 
about teaching on the survey and their teacher log scores. Fifth, there were no significant 
correlations between teachers’ subject matter knowledge on the survey and their teacher log 
scores, but lack of subject matter knowledge was nearly negatively correlated with teachers’ 
mathematical focus scores (p = .13). This means that teachers who lacked subject matter 
knowledge sometimes used the MDM content more than other content. 


Conclusion 

To examine teachers’ implementation fidelity to the MDM curriculum, a new tool, teacher 
logs, were used. These results showed that teachers were generally faithful to the authors’ 
mathematical and pedagogical storylines. This is not surprising as these teachers volunteered to 
implement the MDM curriculum and to take part in the study. However, there was variety shown 
both within and among the teachers. These results support previous research on implementation 
fidelity (e.g., Crawford, Carpenter, Wilson, Schmeister, & McDonald, 2012; Remillard & 
Bryans, 2004; Stein & Kaufman, 2010), and we extended this research by showing results in the 
context of the MDM curriculum. 

To investigate the connection between teachers’ conceptions and their implementation 
fidelity of the MDM curriculum, a survey was used, where the survey results were compared to 
the teacher log results. These results showed that teachers’ beliefs about math were significantly 
positively correlated with their mathematical adherence; thus, teachers with nontraditional views 
of math tended to implement the content of the MDM curriculum—a nontraditional 
curriculum—with high fidelity. According to previous research, how teachers view the nature of 
mathematics, its origins, and its usefulness influence their curricular decisions (e.g., Collopy, 
2003). Stein et al. (2007) explained that sometimes curricular materials “offer views of 
mathematics that conflict with those typically held in mainstream culture” (p. 353). Thus, 
teachers may stray from the written curriculum if the goals do not align with their beliefs about 
math. The results of this study confirmed that if the goals align, adherence is stronger. 

In addition, the results showed that teachers’ beliefs about students were significantly 
positively correlated with their pedagogical adherence and pedagogical focus scores; that is, 
teachers with positive beliefs about students and their abilities tended to support and to not go 
against the authors’ pedagogical storyline. Alternatively, teachers with negative beliefs about 
students and their abilities tended to have lower pedagogical adherence and focus scores. Past 
researchers showed how teachers’ beliefs about students play a role in their instructional 
decisions (e.g., Arbaugh et al., 2006; Remillard & Bryans, 2004). In particular, Arbaugh et al. 
(2006) found results similar to those in the present study, where teachers with negative beliefs 
about students and their abilities often strayed from the authors’ intentions. However, many 
teachers in their study with such beliefs strayed from the intended content while the MDM 
teachers primarily strayed from the intended instructional strategies. Thus, the results from the 
present study complement and add to the research connecting teachers’ beliefs about students to 
their implementation fidelity. 

The results of this study have implications for many. For example, curriculum developers 
need to be clear about intentions and provide support for teachers. Teacher professional 
developers need to be aware of how teachers implement curricula and the conceptions that 
influence their implementations. Finally, future researchers should utilize the new, innovative 
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tool from this study: teacher logs. This tool is a powerful way to quantitatively study 
implementation fidelity. 
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